Handling concept drift in preference learning for interactive decision making

Interactive decision making methods use preference information from the decision maker during the optimization task to guide the search towards favourite solutions. In real-life applications, unforeseen changes in the preferences of the decision maker have to be considered. To the best of our knowledge, no interactive decision making technique has been explicitly designed to recognize and handle preference drift. This paper aims at covering this gap, by extending the Brain-Computer Evolutionary Multi-Objective Optimization (BC-EMO) algorithm to handle preference drift. BC-EMO is a recent multi-objective genetic algorithm. It exploits user judgments of couples of solutions to build incremental models of the user value function. The learnt model is used to refine the genetic population, generating the new individuals in the region of the Pareto front surrounding the favourite solution of the decision maker. The proposed extension of BC-EMO detects the changes of the user preferences by observing the decrease of prediction accuracy of the learnt model. The preference drift is jointly tackled by the BC-EMO learning phase, by a discounting policy for outdated training examples, and by the BC-EMO search phase, by encouraging diversification in the genetic population. Experimental results for a representative preference drift scenario are presented.